{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "307804a3-c02b-4a57-ac0d-172c30ddc851",
            "metadata": {},
            "source": [
                "# Pinecone Index Demo"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "f7010b1d-d1bb-4f08-9309-a328bb4ea396",
            "metadata": {},
            "source": [
                "#### Creating a Pinecone Index"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "d48af8e1",
            "metadata": {},
            "outputs": [],
            "source": [
                "import logging\n",
                "import sys\n",
                "\n",
                "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
                "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "id": "0ce3143d-198c-4dd2-8e5a-c5cdf94f017a",
            "metadata": {},
            "outputs": [],
            "source": [
                "import pinecone"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "id": "4ad14111-0bbb-4c62-906d-6d6253e0cdee",
            "metadata": {},
            "outputs": [],
            "source": [
                "api_key = \"api_key\"\n",
                "pinecone.init(api_key=api_key, environment=\"us-east1-gcp\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "c2c90087-bdd9-4ca4-b06b-2af883559f88",
            "metadata": {},
            "outputs": [],
            "source": [
                "# dimensions are for text-embedding-ada-002\n",
                "pinecone.create_index(\"quickstart\", dimension=1536, metric=\"euclidean\", pod_type=\"p1\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "id": "667f3cb3-ce18-48d5-b9aa-bfc1a1f0f0f6",
            "metadata": {},
            "outputs": [],
            "source": [
                "pinecone_index = pinecone.Index(\"quickstart\")"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "8ee4473a-094f-4d0a-a825-e1213db07240",
            "metadata": {},
            "source": [
                "#### Load documents, build the GPTPineconeIndex"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "id": "0a2bcc07",
            "metadata": {},
            "outputs": [],
            "source": [
                "from gpt_index import GPTPineconeIndex, SimpleDirectoryReader\n",
                "from IPython.display import Markdown, display"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "id": "68cbd239-880e-41a3-98d8-dbb3fab55431",
            "metadata": {},
            "outputs": [],
            "source": [
                "# load documents\n",
                "documents = SimpleDirectoryReader('../paul_graham_essay/data').load_data()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "ba1558b3",
            "metadata": {},
            "outputs": [],
            "source": [
                "# initialize without metadata filter\n",
                "index = GPTPineconeIndex(documents, pinecone_index=pinecone_index)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "ee6eeecb-d54f-4a71-b5fe-0cda8a5c3e10",
            "metadata": {},
            "outputs": [],
            "source": [
                "# [optional] initialize with metadata filters\n",
                "# can define filters specific to this vector index (so you can\n",
                "# reuse pinecone indexes)\n",
                "metadata_filters = {\"title\": \"paul_graham_essay\"}\n",
                "\n",
                "index = GPTPineconeIndex(documents, pinecone_index=pinecone_index, metadata_filters=metadata_filters)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "04304299-fc3e-40a0-8600-f50c3292767e",
            "metadata": {},
            "source": [
                "#### Query Index"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "35369eda",
            "metadata": {},
            "outputs": [],
            "source": [
                "# set Logging to DEBUG for more detailed outputs\n",
                "response = index.query(\"What did the author do growing up?\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "id": "bedbb693-725f-478f-be26-fa7180ea38b2",
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/markdown": [
                            "<b>\n",
                            "\n",
                            "The author grew up writing short stories and programming on an IBM 1401. He also nagged his father to buy him a TRS-80 microcomputer, which he used to write simple games, a program to predict how high his model rockets would fly, and a word processor. He studied philosophy in college, but found it boring and switched to AI. He reverse-engineered SHRDLU for his undergraduate thesis and wrote a book about Lisp hacking while in grad school. He also worked on teaching himself Lisp, which he used to create a SHRDLU clone for his thesis. During the first year of grad school, he realized that AI, as practiced at the time, was a hoax and that there was an unbridgeable gap between what these programs could do and actually understanding natural language. He looked around to see what he could salvage from the wreckage of his plans, and there was Lisp. He decided to focus on Lisp and wrote a book about Lisp hacking, On Lisp, which was published in 1993. He also wanted to build things, so he pursued a career in Computer Science, an uneasy alliance between two halves, theory and systems.</b>"
                        ],
                        "text/plain": [
                            "<IPython.core.display.Markdown object>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "display(Markdown(f\"<b>{response}</b>\"))"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.10.10"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}